Issue:3933 -Multi-node setup for local gradle run #3958

pranikum · 2022-07-20T05:53:23Z

The PR contain changes for enhancing gradle run command to support multinode cluster on local. This change also take in zone count as project parameter for the gradle command. The nodes will be automatically distributed in zones. This will help us in testing multi node setup in local with debugging facilities.

Issues Resolved

#3933

Execute ./gradlew run -PnumNodes=3 -PnumZones=3 to run 3 node cluster with 3 zones.

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions · 2022-07-20T05:59:00Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/815/
CommitID: 599995291ce8311239c2a6983b3fd9d4bd1e6b0f

github-actions · 2022-07-20T11:45:35Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/816/
CommitID: 37b1f20e7f972a1fc834df319b3bb33358b7f1f2

gbbafna · 2022-07-20T12:29:04Z

gradle/run.gradle

+  if (project.hasProperty('nodeCount')) {
+    useCluster testClusters.runMultiNodeTask;


this can disrupt other plugins which use the same property. Should we use zoneCount here and have runMultiZoneTask instead ?

+1 This sounds like a zone aware task. I would rather have it as runMultiZoneTask
Also curious, you've defined both tasks as runMultiNodeTask. Isn't gradle yelling?
Thinking about probably these tasks are unnecessary and we can fold them to one task. Dropped another comment.

gbbafna · 2022-07-20T12:35:25Z

buildSrc/src/main/java/org/opensearch/gradle/testclusters/OpenSearchNode.java

@@ -1239,6 +1243,7 @@ private void createConfiguration() {
        baseConfig.put("path.logs", confPathLogs.toAbsolutePath().toString());
        baseConfig.put("path.shared_data", workingDir.resolve("sharedData").toString());
        baseConfig.put("node.attr.testattr", "test");
+        baseConfig.put("node.attr.availabilityzone", zone);


we should put config only for multi zone clusters .
Instead of passing this , can we use project.findProperty instead here ?

Sure .. Let me try that.

gradle/run.gradle

saratvemulapalli

Lets fold them to one task and make it configurable. This will not conflict existing tasks defined by plugins.

Signed-off-by: Pranit Kumar <[email protected]>

github-actions · 2022-07-21T13:54:14Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/868/
CommitID: 297e57f

Signed-off-by: Pranit Kumar <[email protected]>

github-actions · 2022-07-21T14:15:06Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/867/
CommitID: 2db4aa4

github-actions · 2022-07-21T14:35:48Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/871/
CommitID: e9f3506

saratvemulapalli

These are nice additions, is there a way we could test them?

Signed-off-by: Pranit Kumar <[email protected]>

pranikum · 2022-07-22T14:16:40Z

These are nice additions, is there a way we could test them?

At present couldn't think of an approach to test it. Also don't see ant test around single node run command.

github-actions · 2022-07-22T14:28:55Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/910/
CommitID: 6ef62d3

buildSrc/src/main/java/org/opensearch/gradle/testclusters/OpenSearchCluster.java

reta · 2022-07-25T17:47:14Z

buildSrc/src/main/java/org/opensearch/gradle/testclusters/OpenSearchNode.java

@@ -1239,6 +1243,9 @@ private void createConfiguration() {
        baseConfig.put("path.logs", confPathLogs.toAbsolutePath().toString());
        baseConfig.put("path.shared_data", workingDir.resolve("sharedData").toString());
        baseConfig.put("node.attr.testattr", "test");
+        if (this.project.findProperty("numZones") != null) {


Why we need to check numZones? if (zone != null && !zone.isBlank()) { should be sufficient, right?

Since we are passing zone value for the first node as well. Which will be true for the scenario where -PnumZones will not be set. Hence wanted to check the existence of the property. instead of zone variable.
Don't want to add zone attribute for the single node cluster. i.e
./gradlew run .

https://github.com/opensearch-project/OpenSearch/pull/3958/files#diff-a22e2ce210c503bbd47be71cc6e770064e8a9e008738d28fee5b381918f1fb8cR109

The decision should be made in OpenSearchCluster: this class knows zones configuration (and should either associated node with zone or not).

Moved the decision making to OpensearchCluster class.

…n OpensearchNode Signed-off-by: Pranit Kumar <[email protected]>

github-actions · 2022-07-27T14:44:18Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/1041/
CommitID: 8a18a9f

buildSrc/src/main/java/org/opensearch/gradle/testclusters/OpenSearchCluster.java

Bukhtawar

Why call it numZones. All through the code base we use a generic term awareness attribute which could mean a rack/data centre/zone/switch

Signed-off-by: Pranit Kumar <[email protected]>

github-actions · 2022-07-28T13:47:19Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/1067/
CommitID: 5e1bc81

pranikum · 2022-07-28T13:59:57Z

Why call it numZones. All through the code base we use a generic term awareness attribute which could mean a rack/data centre/zone/switch

I checked the code again . I see we are using term zone for awareness attribute. Couldn't think of any better way of classification.

https://github.com/opensearch-project/OpenSearch/blob/main/server/src/main/java/org/opensearch/cluster/routing/allocation/decider/AwarenessAllocationDecider.java#L85

saratvemulapalli · 2022-07-28T21:13:01Z

@pranikum looks like precommit is failing.
Could you take a look?

Execution failed for task ':build-tools:spotlessJavaCheck'.
> The following files had format violations:
      src/main/java/org/opensearch/gradle/testclusters/OpenSearchCluster.java
          @@ -149,7 +149,7 @@
           ············}
           ········}·else·{
           ············for·(int·i·=·nodes.size();·i·<·numberOfNodes;·i++)·{
          -················addNode(clusterName·+·"-"·+·i,·""·);
          +················addNode(clusterName·+·"-"·+·i,·"");
           ············}
           ········}
           ····}

You can run './gradlew :build-tools:spotlessApply' to fix these violations.

Signed-off-by: Pranit Kumar <[email protected]>

github-actions · 2022-07-29T12:16:34Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/1104/
CommitID: a131f84

github-actions · 2022-07-29T13:18:41Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/1106/
CommitID: cf7beda

codecov-commenter · 2022-07-29T13:21:52Z

Codecov Report

Merging #3958 (9992f81) into main (fbe93d4) will decrease coverage by 0.05%.
The diff coverage is 0.00%.

@@             Coverage Diff              @@
##               main    #3958      +/-   ##
============================================
- Coverage     70.55%   70.49%   -0.06%     
+ Complexity    57013    56974      -39     
============================================
  Files          4603     4603              
  Lines        274478   274496      +18     
  Branches      40201    40206       +5     
============================================
- Hits         193658   193515     -143     
- Misses        64511    64679     +168     
+ Partials      16309    16302       -7

Impacted Files	Coverage Δ
...nsearch/gradle/testclusters/OpenSearchCluster.java	`0.00% <0.00%> (ø)`
...opensearch/gradle/testclusters/OpenSearchNode.java	`0.00% <0.00%> (ø)`
...gations/bucket/composite/CompositeAggregation.java	`15.00% <0.00%> (-85.00%)`	⬇️
...pensearch/client/cluster/RemoteConnectionInfo.java	`0.00% <0.00%> (-68.30%)`	⬇️
...a/org/opensearch/client/cluster/SniffModeInfo.java	`0.00% <0.00%> (-52.95%)`	⬇️
...a/org/opensearch/tasks/TaskCancelledException.java	`50.00% <0.00%> (-50.00%)`	⬇️
...opensearch/persistent/PersistentTasksExecutor.java	`22.22% <0.00%> (-44.45%)`	⬇️
.../org/opensearch/client/indices/AnalyzeRequest.java	`31.00% <0.00%> (-43.00%)`	⬇️
...aggregations/bucket/composite/ParsedComposite.java	`44.44% <0.00%> (-40.75%)`	⬇️
.../opensearch/client/cluster/RemoteInfoResponse.java	`61.53% <0.00%> (-38.47%)`	⬇️
... and 488 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

pranikum · 2022-07-30T03:29:27Z

@pranikum looks like precommit is failing. Could you take a look?

Execution failed for task ':build-tools:spotlessJavaCheck'.
> The following files had format violations:
      src/main/java/org/opensearch/gradle/testclusters/OpenSearchCluster.java
          @@ -149,7 +149,7 @@
           ············}
           ········}·else·{
           ············for·(int·i·=·nodes.size();·i·<·numberOfNodes;·i++)·{
          -················addNode(clusterName·+·"-"·+·i,·""·);
          +················addNode(clusterName·+·"-"·+·i,·"");
           ············}
           ········}
           ····}

You can run './gradlew :build-tools:spotlessApply' to fix these violations.

Done.

Signed-off-by: pranikum <[email protected]>

pranikum · 2022-08-05T07:09:55Z

@Bukhtawar , Can we please merge this PR.

github-actions · 2022-08-05T07:14:00Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/1482/
CommitID: 4348da1

github-actions · 2022-08-05T07:34:11Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/1483/
CommitID: 9992f81

…#3958) * Issue:3933 - Multinode setup for local gradle run Signed-off-by: Pranit Kumar <[email protected]>

pranikum requested review from a team and reta as code owners July 20, 2022 05:53

gbbafna reviewed Jul 20, 2022

View reviewed changes

saratvemulapalli reviewed Jul 20, 2022

View reviewed changes

gradle/run.gradle Show resolved Hide resolved

saratvemulapalli requested changes Jul 20, 2022

View reviewed changes

pranikum added 2 commits July 21, 2022 19:17

Issue:3933 - Initial Checkin for Multinode setup for local gradle run

1f54bd2

Signed-off-by: Pranit Kumar <[email protected]>

Issue:3933 - Fix spotless check failure

2db4aa4

Signed-off-by: Pranit Kumar <[email protected]>

pranikum force-pushed the multi_node_gradle_run_support branch from 37b1f20 to 2db4aa4 Compare July 21, 2022 13:48

Issue:3933 :- Update property name.

297e57f

Signed-off-by: Pranit Kumar <[email protected]>

Issue:3933: Use default numNodes as 3 for multiNode task.

e9f3506

Signed-off-by: Pranit Kumar <[email protected]>

saratvemulapalli reviewed Jul 21, 2022

View reviewed changes

pranikum changed the title ~~Issue:3933 - Initial Checkin for Multinode setup for local gradle run~~ Issue:3933 -Multi-node setup for local gradle run Jul 21, 2022

Issue:3933 : Use same confguration for single node and multi node run.

6ef62d3

Signed-off-by: Pranit Kumar <[email protected]>

reta reviewed Jul 25, 2022

View reviewed changes

buildSrc/src/main/java/org/opensearch/gradle/testclusters/OpenSearchCluster.java Show resolved Hide resolved

reta reviewed Jul 25, 2022

View reviewed changes

pranikum requested a review from saratvemulapalli July 26, 2022 04:43

pranikum added 2 commits July 27, 2022 10:12

Issue:3933: Merge branch 'main' into multi_node_gradle_run_support

d3a8610

Issue:3933: check for project property in OpensearchCluster instead o…

8a18a9f

…n OpensearchNode Signed-off-by: Pranit Kumar <[email protected]>

reta reviewed Jul 27, 2022

View reviewed changes

buildSrc/src/main/java/org/opensearch/gradle/testclusters/OpenSearchCluster.java Outdated Show resolved Hide resolved

Bukhtawar reviewed Jul 27, 2022

View reviewed changes

Issue:3933: Branch off zone vs without zone flow

5e1bc81

Signed-off-by: Pranit Kumar <[email protected]>

reta approved these changes Jul 28, 2022

View reviewed changes

saratvemulapalli approved these changes Jul 28, 2022

View reviewed changes

Issue-3933: Spotless java check

a131f84

Signed-off-by: Pranit Kumar <[email protected]>

Issue-3933: Merge branch 'main' into multi_node_gradle_run_support

cf7beda

pranikum added 2 commits August 5, 2022 12:32

Add awareness attribute for zone. Use zone instead of AvailablityZone.

4348da1

Signed-off-by: pranikum <[email protected]>

Merge branch 'main' into multi_node_gradle_run_support

9992f81

Bukhtawar approved these changes Aug 5, 2022

View reviewed changes

Bukhtawar merged commit 24527fc into opensearch-project:main Aug 5, 2022

dreamer-89 pushed a commit to dreamer-89/OpenSearch that referenced this pull request Aug 12, 2022

Issue:3933 -Multi-node setup for local gradle run (opensearch-project…

36dd103

…#3958) * Issue:3933 - Multinode setup for local gradle run Signed-off-by: Pranit Kumar <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Issue:3933 -Multi-node setup for local gradle run #3958

Issue:3933 -Multi-node setup for local gradle run #3958

pranikum commented Jul 20, 2022 •

edited

Loading

github-actions bot commented Jul 20, 2022

github-actions bot commented Jul 20, 2022

gbbafna Jul 20, 2022

saratvemulapalli Jul 20, 2022 •

edited

Loading

gbbafna Jul 20, 2022

pranikum Jul 21, 2022

pranikum Jul 25, 2022

saratvemulapalli left a comment

github-actions bot commented Jul 21, 2022

github-actions bot commented Jul 21, 2022

github-actions bot commented Jul 21, 2022

saratvemulapalli left a comment

pranikum commented Jul 22, 2022

github-actions bot commented Jul 22, 2022

reta Jul 25, 2022

pranikum Jul 26, 2022 •

edited

Loading

reta Jul 26, 2022

pranikum Jul 27, 2022

reta Jul 27, 2022

github-actions bot commented Jul 27, 2022

Bukhtawar left a comment

github-actions bot commented Jul 28, 2022

pranikum commented Jul 28, 2022

saratvemulapalli commented Jul 28, 2022

github-actions bot commented Jul 29, 2022

github-actions bot commented Jul 29, 2022

codecov-commenter commented Jul 29, 2022 •

edited

Loading

pranikum commented Jul 30, 2022

pranikum commented Aug 5, 2022

github-actions bot commented Aug 5, 2022

github-actions bot commented Aug 5, 2022

		if (project.hasProperty('nodeCount')) {
		useCluster testClusters.runMultiNodeTask;

Issue:3933 -Multi-node setup for local gradle run #3958

Issue:3933 -Multi-node setup for local gradle run #3958

Conversation

pranikum commented Jul 20, 2022 • edited Loading

Issues Resolved

Check List

github-actions bot commented Jul 20, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Jul 20, 2022

Gradle Check (Jenkins) Run Completed with:

Choose a reason for hiding this comment

saratvemulapalli Jul 20, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

saratvemulapalli left a comment

Choose a reason for hiding this comment

github-actions bot commented Jul 21, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Jul 21, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Jul 21, 2022

Gradle Check (Jenkins) Run Completed with:

saratvemulapalli left a comment

Choose a reason for hiding this comment

pranikum commented Jul 22, 2022

github-actions bot commented Jul 22, 2022

Gradle Check (Jenkins) Run Completed with:

Choose a reason for hiding this comment

pranikum Jul 26, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Jul 27, 2022

Gradle Check (Jenkins) Run Completed with:

Bukhtawar left a comment

Choose a reason for hiding this comment

github-actions bot commented Jul 28, 2022

Gradle Check (Jenkins) Run Completed with:

pranikum commented Jul 28, 2022

saratvemulapalli commented Jul 28, 2022

github-actions bot commented Jul 29, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Jul 29, 2022

Gradle Check (Jenkins) Run Completed with:

codecov-commenter commented Jul 29, 2022 • edited Loading

Codecov Report

pranikum commented Jul 30, 2022

pranikum commented Aug 5, 2022

github-actions bot commented Aug 5, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Aug 5, 2022

Gradle Check (Jenkins) Run Completed with:

pranikum commented Jul 20, 2022 •

edited

Loading

saratvemulapalli Jul 20, 2022 •

edited

Loading

pranikum Jul 26, 2022 •

edited

Loading

codecov-commenter commented Jul 29, 2022 •

edited

Loading